feat: accept an external consumption forecast as a time-series entity - #437
feat: accept an external consumption forecast as a time-series entity#437johanzander wants to merge 4 commits into
Conversation
…#428) BESS optimizes against forecasts but never modelled loads itself — for consumption every existing strategy was either a constant or an in-house statistical model, none able to express a known, shaped load (an EV session, weather-driven aircon) the user can predict but a trimmed-mean baseline deliberately discounts as an outlier. Adds `ha_consumption_series`: a new strategy reading a user-authored HA entity with `raw_today`/`raw_tomorrow` timestamped {start, value} attributes, mirroring the existing price-manager pattern rather than solar's hour-indexed parser (which structurally can't represent sub-hourly data). Accepts 15- or 60-minute record spacing, normalized onto the DP's quarter-hour grid; a missing, stale, malformed, or horizon-short series is an explicit ConsumptionForecastUnavailableError, never a silent fallback to a flat profile — unlike ha_statistics's fixed-value fallback on insufficient data. Also renames the scalar `sensor` strategy's misleading "Consumption Forecast" UI category to "Consumption Average (48h Rolling)", since it was never a real forecast, and adds an example HA template so the seam is usable without starting from a blank YAML file. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ption-forecast-series # Conflicts: # CHANGELOG.md
…ption-forecast-series # Conflicts: # CHANGELOG.md
Still conflicted — merge attempted twice today, both abortedThis PR is
The conflicts
These are logic conflicts, not mechanical ones. Resolving them means preserving both this branch's external-consumption-forecast feature and ~2 weeks of One trap worth knowing before you startThe worktree accumulated 150 untracked files byte-identical to comm -12 <(git ls-files --others --exclude-standard | sort) \
<(git ls-tree -r origin/main --name-only | sort)Then delete only the entries that are byte-identical to |
…e-argued (#636) * feat: put PRs on the board so a decision about one is recorded, not re-argued The board held issues only, so every judgement about a PR had nowhere to live. "#167 and #354 are blocked", "#437 and #490 are lower priority, later" were real decisions the maintainer had already made, and the rhythm pass re-reported all four as due on every tick because nothing recorded them. The same conversation happened every 30 minutes. Projects v2 takes PRs as items with the identical field set, so the fix is membership rather than a parallel mechanism. A PR card carries the same `Priority` and `Awaiting` an issue card does; the digest emits them as `pr_board`, and `backlog-rhythm.sh` joins by number and suppresses on them: an `Awaiting` means parked on someone, `P4` means later-not-never. `content.type` is what separates PR cards from issue ones, confirmed against a real card rather than assumed — an added PR reports "PullRequest" with number/title/url/repository alongside it. Numbers are unique across issues and PRs in one repository, so this cannot collide with the existing issue lookup. Suppressed PRs are COUNTED AND LISTED, never dropped: the pass ends with `deferred: 4 (#490 priority P4; #167 awaiting discussion; ...)`. Silently vanishing would trade one failure for another — the goal is to stop re-asking about a settled decision, not to lose the item. CONTRACT CHANGE: an APPROVED, green, still-draft PR is now its own action, `mark_ready`, and it is the one thing no board decision can defer. It used to hand back to `implement-issue` like any other unfinished draft, on the principle that this pass must not grow a second review loop. That principle still holds, but it is what left #629 sitting approved, green and draft: the remedy on offer was a whole `implement-issue` session, and nobody spends one of those to run a single command. `gh pr ready` is a terminal action, not a loop, so naming it here duplicates nothing. `awaiting_maintainer` is deliberately NOT carved out the same way. An approved PR waiting on a merge is not broken; it is the maintainers call when to take it, and P4 is exactly how they say later. #490 sat approved for a day and was reported every tick as though that were news. Live effect: 31 actions -> 27, with one `deferred: 4` line in place of four recurring items, and #631 correctly still reported as merge-ready. No CHANGELOG entry: agent tooling, no user-visible effect. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012LExo6fcbup75vtc9NfoAR * fix: mark_ready needs green checks, not just a clean merge Found by running the rule against the live fleet on its first pass, which is the only reason it was caught before the PR merged. `mergeable` reports whether the branch merges cleanly and nothing else, so it reads MERGEABLE while CI is still running or has failed outright. #633 was APPROVED, MERGEABLE and had Algorithm tests and E2E still IN_PROGRESS, and the rule duly reported "gh pr ready 633 — then it is the maintainers to merge". GitHub itself disagreed: `mergeStateStatus` was BLOCKED. Flipping a red or pending PR out of draft is worse than leaving it there. `ready` is supposed to mean the maintainer can merge without checking anything else, and that claim is the only thing making the flag worth setting. So `mark_ready` now also requires every check to have concluded SUCCESS, SKIPPED or NEUTRAL. SKIPPED is green on purpose: this repo path-filters Algorithm tests and Docker build, so every backend-only PR skips them and treating that as not-green would withhold the action from almost everything. An empty rollup is green too — a PR with no checks configured has nothing failing. The deferred list mirrors the same condition, so an approved-but-pending PR with a P4 card appears in exactly one place rather than both. Live effect: #633 falls back to resume_implementation until its CI settles, which is the correct answer and the one the first version got wrong. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012LExo6fcbup75vtc9NfoAR --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
… branch Two halves of the same failure: work landing outside a worktree, and nobody noticing when it does. ## The hook CLAUDE.md has said "never edit any file on main, even a one-line doc fix" unconditionally for a long time, and it keeps being skipped. The reason is structural, not carelessness: it is prose, so it has to be REMEMBERED at the moment of the first edit — and that is exactly the moment a session which opened as a question has no reason to reconsider it. Six live sessions currently sit in the main checkout for perfectly good read-only reasons; nothing catches the one that quietly starts editing. check-worktree-path.sh already guarded CROSS-checkout edits and passed same-checkout ones, so main-to-main sailed through. It now also refuses any edit made from the main checkout, detected by --git-dir equalling --git-common-dir. That is a path comparison, the only shape docs/agents/rules.md sanctions here — it never guesses what a command will touch. Linked worktrees and sibling checkouts both differ, so both still work; the rule is "be in a worktree", not "be under .claude/". The denial names the remedy (EnterWorktree) and says what the main checkout still does — questions, gh, backlog, dispatch — because a block without a next move gets worked around. Residual gap, stated plainly: this governs Edit/Write/NotebookEdit. A Bash `sed -i` still writes. Guarding that would mean parsing command strings, which rules.md forbids for this hook and which has produced false positives here four times. ## The detector pr-state.sh gains a local-writer section. GitHub cannot see this: a branch with two writers looks normal through the API, because the divergence exists only between a local checkout and the remote and it collapses into an ordinary merge the moment someone reconciles. #619 is the worked example. One writer took the branch at 08:09 and worked from that base; another pushed 23031e7 at 09:34. The reviewer reviewed 23031e7 three times, twice with blocking findings, while the first line never held that commit. Fifteen hours later it landed as `Merge remote-tracking branch 'origin/fix/...' into fix/...` — a branch merged into itself, which is the fingerprint. `git rev-list --left-right` would have caught it at 09:34. Run against the live fleet it also surfaces the precursor state: #437 is 5 commits behind its own remote and #614 is 3, so a commit in either worktree diverges immediately. Skipped LOUDLY outside a checkout, since this script is also meant for a container fleet where a silent skip would read as "no divergence found". Verified by mutation: disabling the main-checkout guard reddens 2 of the 6 hook tests. The divergence detector is tested against a real two-clone scenario, not fixtures. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Parked: not to be merged yet — awaiting user feedbackMaintainer decision, recorded here so it stops being re-derived every backlog pass. Board fields set to match:
Together those suppress this PR from the rhythm pass's action list. It is still counted and listed as deferred, so it stays findable — it is parked, not dropped. Do not resolve the conflicts yet. This PR has been
Two attempts have already been spent on them — a Unpark by clearing |
Summary
ha_consumption_series: a new consumption-forecast strategy reading a user-authored HA time-series entity (raw_today/raw_tomorrowattributes of timestamped{start, value}records), mirroring the existing price-manager entity pattern rather than solar's hour-indexed parser (which can't represent sub-hourly data).ConsumptionForecastUnavailableError— this strategy never silently falls back to a flat profile, unlikeha_statistics's fixed-value fallback on insufficient data.sensorstrategy's misleading "Consumption Forecast" UI category to "Consumption Average (48h Rolling)" (it was never a real forecast) and adds a new "Consumption Forecast Series" category for the new entity.docs/USER_GUIDE.mdso the seam is usable without starting from a blank YAML file.Root cause
From the issue: every existing consumption strategy is either a constant or an in-house statistical model.
ha_statistics's trimmed mean deliberately discounts one-off spikes (the right call for outlier robustness) but as a side effect is permanently blind to a known, predictable load like an EV session — nothing puts that load back into the forecast. Solar has an external-forecast seam (Solcast); consumption had none.Fix
HomeAssistantAPIController.get_consumption_forecast_series()/get_consumption_forecast_series_tomorrow()(core/bess/ha_api_controller.py) — fetch and normalize the entity, mirroringprice_manager.py'sraw_today/raw_tomorrowparsing.BatterySystemManager._get_consumption_forecastdispatches to it for the new strategy;_extend_consumption_predictions(extracted from the existing tomorrow-horizon-extension logic) uses the entity's realraw_tomorrowdata instead of repeating today's pattern when the DP horizon spans midnight — the same treatment solar's_fetch_tomorrow_solar_forecastalready gets.sensor_collector.check_prediction_health) and the strategy-comparison endpoint (get_consumption_forecast_comparison) both cover the new strategy.HomeFormSection.tsx, sensor category rename + addition insensorDefinitions.ts, updatedSensorConfigSection.tsxdetection logic,ConsumptionForecastComparison.tsxchart labels/colors.Test plan
./scripts/quality-check.shpasses locally (fast suite: 1399 passed, 15 skipped; frontend: 103 passed; Black/Ruff/TypeScript/ESLint clean).venv/bin/pytest -m slowpasses (394 passed, 3 skipped)docker-compose.ci.yml(real backend + mock-HA): configured the new strategy and a shaped entity via the realPATCH /api/settingsAPI, confirmedGET /api/consumption-forecast-comparisonreturns the correct shaped hourly profile (EV hour at 16.0 kWh vs 1.2 kWh baseline) through the live system, confirmed the health check reportsOKwith the exact shaped series and then a clearERROR(no silent fallback) when the entity'sraw_todaydata was removed, and confirmed the new frontend strings are present in the actual served JS bundleCloses #428